Tea: A High-level Language and Runtime System for Automating Statistical Analysis

Though statistical analyses are centered on research questions and hypotheses, current statistical analysis tools are not. Users must first translate their hypotheses into specific statistical tests and then perform API calls with functions and parameters. To do so accurately requires that users have statistical expertise. To lower this barrier to valid, replicable statistical analysis, we introduce Tea, a high-level declarative language and runtime system. In Tea, users express their study design, any parametric assumptions, and their hypotheses. Tea compiles these high-level specifications into a constraint satisfaction problem that determines the set of valid statistical tests, and then executes them to test the hypothesis. We evaluate Tea using a suite of statistical analyses drawn from popular tutorials. We show that Tea generally matches the choices of experts while automatically switching to non-parametric tests when parametric assumptions are not met. We simulate the effect of mistakes made by non-expert users and show that Tea automatically avoids both false negatives and false positives that could be produced by the application of incorrect statistical tests.Comment: 11 page

Principled Optimization Of Dynamic Neural Networks

Thesis (Ph.D.)--University of Washington, 2020In the past decade deep learning has revolutionized many areas of computer science.Deep learning computations mostly consist of expensive linear algebra kernels defined over a mixture of large sparse and dense tensors. From the early days of deep learning framework development, researchers realized the potential for applying compiler optimizations to accelerate neural networks. As deep learning continues to grow in popularity the diversity of models also continues to grow. Due to the early success of deep learning in computer vision, early deep learning systems were were focused on static, feed-forward networks processing fixed sized images. First-generation deep learning compilers have also been similarly overfit for static model compilation, with strong assumptions of static control-flow, static tensor dimensions and no complex data structures. A focus on static models has created challenges for deep learning practitioners, as dynamic models introduce input-dependent graph topology, violating key invariants of existing systems and invalidating optimizations designed for purely static data flow graphs. This lack of support has manifested as series of ad-hoc extensions to both frameworks, deep learning runtimes and compilers. Choosing to ignore dynamic behaviors has allowed deep learning compilers to make significant strides in optimizing common deep learning workloads, but existing techniques miss increasing generality without sacrificing performance. This dissertation in particular focuses on an under served, yet important problem: the representation, optimization, differentiation and execution of dynamic neural networks.In this thesis I propose generalizing overspecialized compilation techniques applied to static dataflow graphs, the predominant programming model of deep learning, to fully dynamic neural networks. These generalizations are powered by a simple insight: dynamic neural networks are just programs which manipulate tensors. The challenge is constructing a representation that captures this generality in a principled manner, while not sacrificing state-of-the-art performance or the programming model. In particular, the contributions include: an intermediate representation which can represent dynamic behaviors, a new automatic differentiation technique for dynamic neural networks, a set of general optimizations which work on all programs, as well as specialized dynamic optimizations, an efficient runtime for dynamic neural networks. The efforts of my thesis now exists in Apache TVM, a deep learning compiler framework.Apache TVM is deployed in production at multiple leading companies including Amazon, Facebook, and Microsoft and is a critical piece of the technology stack at OctoML a company I co-founded around the TVM project. One notable impact is its use in Amazon Alexa, Amazon's AI assistant which executes on a variety of devices such as ``smart speakers'' which include digital assistants. Amazon engineers used Relay to optimize Alexa’s wake word model, executed each time a user interacts with Alexa

Improved Type Specialization for Dynamic Scripting Languages

Type feedback and type inference are two common methods used to optimize dynamic languages such as JavaScript. Each of these methods has its own strengths and weaknesses, and we propose that each can benefit from the other if combined in the right way. We explore the interdependency between these two methods and propose two novel ways to combine them in order to significantly increase their aggregate benefit and decrease their aggregate overhead. In our proposed strategy, an initial type inference pass is applied that can reduce type feedback overhead by enabling more intelligent placement of profiling hooks. This initial type inference pass is novel in the literature. After profiling, a final type inference pass uses the type information from profiling to generate efficient code. While this second pass is not novel, we significantly improve its effectiveness in a novel way by feeding the type inference pass information about the function signature, i.e., the types of the function’s arguments for a specific function invocation. Our results show significant speedups when using these low-overhead strategies, ranging from 1.2 × to 4 × over an implementation that does not perform type feedback or type inference based optimizations. Our experiments are carried out across a wide range of traditional benchmarks and realistic web applications. The results also show an average reduction of 23.5 % in the size of the profiled data for these benchmarks